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Abstract 

Submodular function maximization is a central problem in combinatorial optimization, gener- 
alizing many important problems including Max Cut in directed/undirected graphs and in hyper- 
graphs, certain constraint satisfaction problems, maximum entropy sampling, and maximum facility 
location problems. Unlike submodular minimization, submodular maximization is NP-hard. In this 
paper, we give the first constant-factor approximation algorithm for maximizing any non-negative 
submodular function subject to multiple matroid or knapsack constraints. We emphasize that our 
results are for non-monotone submodular functions. In particular, for any constant k, we present a 

-approximation for the submodular maximization problem under k matroid constraints, 

and a (i — e) -approximation algorithm for this problem subject to k knapsack constraints (e > is 
any constant). We improve the approximation guarantee of our algorithm to , for k > 2 

partition matroid constraints. This idea also gives a ( iq:;) -approximation for maximizing a mono- 
tone submodular function subject to fc > 2 partition matroids, which improves over the previously 
best known guarantee of 



*IBM T.J. Watson Research Center, jonleeSus . ibm. com 
^Google Research, NYC mirrokni@google . com 
■''Carnegie Mellon University, viswanatSandrew . emu . edu 
''IBM T.J. Watson Research Center, sviri@us . ibm. com 



1 Introduction 



In this paper, we consider the problem of maximizing a nonnegative submodular function f, defined 
on a ground set V, subject to matroid constraints or knapsack constraints. A function / : 2^ — >^ M is 
submodular if for all S,T (ZV, f{S\JT) + f{Sr\T) < f{S) + f{T). Throughout, we assume that our 
submodular function / is given by a value oracle; i.e., for a given set 5" C F, an algorithm can query 
an oracle to find its value f{S). Furthermore, all submodular functions we deal with are assumed to be 
non-negative. We also denote the ground set F = [n] = {1, 2, • • • , n}. 

We emphasize that our focus is on submodular functions that are not required to be monotone (i.e., 
we do not require that f{X) < f{Y) for X C y C V). Non-monotone submodular functions appear 
in several places including cut functions in weighted directed or undirected graphs or even hypergraphs, 
maximum facility location, maximum entropy sampling, and certain constraint satisfaction problems. 

Given a weight vector w for the ground set V, and a knapsack of capacity C, the associated knapsack 
constraint is that the sum of weights of elements in the solution S should not exceed the capacity C, 
i.e, J2jeS — ^- '^^^ usage, we consider k knapsack constraints defined by weight vectors and 
capacities Ci, for i = 1, ... ,k. 

We assume some familiarity with matroids BOl and associated algorithmics P31 . Briefly, for a ma- 
troid A4, we denote the ground set of ^A by £{A4), its set of independent sets by X(A^), and its set of 
bases by 13 {A4). For a given matroid M, the associated matroid constraint is 5 G and the asso- 

ciated matroid base constraint is 5 E B{Ai). In our usage, we deal with k matroids Aii, . . . , M.k on 
the common ground set V := £{Mi) = ■ ■ ■ = £{M-k) (which is also the ground set of our submodular 
function /), and we let 2j := T{Mi) for f = 1, . . . , k. 

Background. Optimizing submodular functions is a central subject in operations research and com- 
binatorial optimization |[36l . This problem appears in many important optimization problems includ- 
ing cuts in graphs |[T9l |4T1 l26l . rank function of matroids (12[ [T6l . set covering problems |[T3l . plant 
location problems lU [TOl [TTJ |2]|, and certain satisfiability problems |[25l [141. and maximum entropy 
sampling |[32l [33ll . Other than many heuristics that have been developed for optimizing these func- 
tions |[20ll2Tll27ll43l[3TI . many exact and constant-factor approximation algorithms are also known for 
this problem |[38l[39ll44l|26l[T5ll48l[T8l . In some settings such as set covering or matroid optimization, 
the relevant submodular functions are monotone. Here, we are more interested in the general case where 
f{S) is not necessarily monotone. 

Unlike submodular minimization B4l|26l . submodular function maximization is NP-hard as it gen- 
eralizes many NP-hard problems, like Max-Cut |[T9l[T4l and maximum facility location |[9l[T0l|2l. Other 
than generalizing combinatorial optimization problems like Max Cut [|19il . Max Directed Cut ll4l l22i . 
hypergraph cut problems, maximum facility location ||2l|9l[T0l, and certain restricted satisfiabiUty prob- 
lems ||25l[T4l . maximizing non-monotone submodular functions have applications in a variety of prob- 
lems, e.g, computing the core value of supermodular games ||46l, and optimal marketing for revenue 
maximization over social networks |[23l . As an example, we describe one important application in the 
statistical design of experiments. The maximum entropy sampling problem is as follows: Let A be the 
n-by-n covariance matrix of a set of Gaussian random variables indexed by [n]. For 5" C [n], let A[S] 
denote the principal submatrix of A indexed by S. It is well known that (up to constants depending on 
\S\), logdetyl[5] is the entropy of the random variables indexed by S. Furthermore, logdet is 
submodular on [n] . In applications of locating environmental monitoring stations, it is desired to choose 
s locations from [n] so as to maximize the entropy of the associated random variables, so that problem 
is precisely one of maximizing a non-monotone submodular function subject to a cardinality constraint. 
Of course a cardinality constraint is just a matroid base constraint for a uniform matroid. We note that 
the entropy function is not even approximately monotone (see QUI ). The maximum entropy sampling 
problem has mostly been studied from a computational point of view, focusing on calculating optimal 
solutions for moderate-sized instances (say n < 200) using mathematical programming methodolo- 



1 



gies (e.g, see |[32l [33l [34l [29l |6l 21), and our results provide the first set of algoritlims witfi provable 
constant-factor approximation guarantee. 

Recently, a | -approximation was developed for maximizing non-negative non-monotone submod- 
ular functions without any side constraints |[T5l . This algorithm also provides a tight | -approximation 
algorithm for maximizing a symmetric' submodular function lITSl . However, the algorithms developed 



in 11151 for non-monotone submodular maximization do not handle any extra constraints. 

For the problem of maximizing a monotone submodular function subject to a matroid or multiple 
knapsack constraints, tight (l — i) -approximation are known |l38l|71|49l|42l|28l. Maximizing monotone 

submodular functions over k matroid constraints was considered in ll39l . where a ^^q^^ -approximation 
was obtained. This bound is cun^ently the best known ratio, even in the special case of partition matroid 
constraints. However, none of these results generalize to non-monotone submodular functions. 

Better results are known either for specific submodular functions or for special classes of matroids. 
A ^-approximation algorithm using local seaixh was designed in [|42| for the problem of maximizing 
a linear function subject to k matroid constraints. Constant factor approximation algorithms are known 
for the problem of maximizing directed cut m or hypergraph cut ||3l subject to a uniform matroid (i.e. 
cardinality) constraint. 

Hardness of approximation results are known even for the special case of maximizing a linear func- 
tion subject to k partition matroid constraints. The best known lower bound is an ^{j^^) hardness 
of approximation [24 1. Moreover, for the unconstrained maximization of non-monotone submodulai^ 
functions, it has been shown that achieving a factor better than ^ cannot be done using a subexponential 
number of value queries |[T5l . 

Our Results. In this paper, we give the first constant-factor approximation algorithms for maximiz- 
ing a non-monotone submodular function subject to multiple matroid constraints, or multiple knapsack 
constraints. More specifically, we give the following new results (below e > is any constant). 

(1) For every constant > 1, we present a ^^-j-^-p-j— ^-approximation algorithm for maximizing any 

non-negative submodular function subject to k matroid constraints (Section |2l). This implies a i^-^^- 
approximation algorithm for maximizing non-monotone submodular functions subject to a single ma- 
troid constraint. Moreover, this algorithm is a -approximation in the case of symmetric submod- 
ular functions. Asymptotically, this result is nearly best possible because there is an hai^dness 
of approximation, even in the monotone case |[24l . 

(2) For every constant A; > 1, we present a — e) -approximation algorithm for maximizing any non- 
negative submodular function subject to a /c-dimensional knapsack constraint (Section [3]). To achieve 
the approximation guarantee, we first give a (| — e) -approximation algorithm for a fractional relaxation 
(similar to the fractional relaxation used in B9l ). We then use a simple randomized rounding tech- 
nique to convert a fractional solution to an integral one. A similar method was recently used in ll28l for 
maximizing a monotone submodular function over knapsack constraints, but neither their algorithm for 
the fractional relaxation, nor their rounding method is directly applicable to non-monotone submodular 
functions. 

(3) For submodular maximization under k > 2 partition matroid constraints, we obtain improved ap- 
proximation guarantees (SectionlU). We give a I — — ^ I -approximation algorithm for maximizing 



non-monotone submodular functions subject to k partition matroids. Moreover, our idea gives a yk^j ' 
approximation algorithm for maximizing a monotone submodular function subject to A; > 2 partition 
matroid constraints. This is an improvement over the previously best known bound of from |[39l . 



'The function / : 2^ — > R is symmetric if for all S C V, f{S) = f{V \ S). For example, cut functions in undirected 
graphs are well-known examples of symmetric (non-monotone) submodular functions 
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(4) Finally, we study submodular maximization subject to a matroid base constraint in Appendix IE] We 
give a (i — e) -approximation in the case of symmetric submodular functions. Our result for general 
submodular functions only holds for special matroids: we obtain a (| — e) -approximation when the ma- 
troid contains two disjoint bases. In particular, this implies a (g — e) -approximation for the problem of 
maximizing any non-negative submodular function subject to an exact cardinality constraint. Previously, 
only special cases of directed cut or hypergraph cut Q subject to an exact cardinality constraint were 
considered. 

Our main technique for the above results is local search. Our local search algorithms are different 
from the previously used variant of local search for unconstrained maximization of a non-negative sub- 
modular function |fT5]], or the local search algorithms used for Max Directed Cut ||4l|22l. In the design 
of our algorithms, we also use structural properties of matroids, a fractional relaxation of submodular 
functions, and a randomized rounding technique. 

2 Matroid Constraints 

In this section, we give an approximation algorithm for submodular maximization subject to k matroid 
constraints. The problem is as follows: Let / be a non-negative submodular function defined on ground 
set V. Let M.i, - • • , 7V4fc be A; arbitrary matroids on the common ground set V. For each matroid M.j 
(with j G [k]) we denote the set of its independent sets by Tj. We consider the following problem: 

max|/(5) : S € n^'^iJ^j . (1) 

We give an approximation algorithm for this problem using value queries to / that runs in time nP^^^ . 
The starting point is the following local search algorithm. Starting with = 0, repeatedly perform one 
of the following local improvements: 

• Delete operation. If e G 5 such that f{S \ {e}) > f{S), then S ^ S\ {e}. 

• Exchange operation. \ide V\S and e S'U{</)} (for 1 < z < A;) are such that (S'\{ei})U{(i} e 
XiforalH G [k] and \ {ei, • • • ,efc})U{d}) > then 5 ^ (5 \ {ei, • • • ,efc})U{d}. 

When dealing with a single matroid constraint (A; = 1), the local operations correspond to: delete an 
element, add an element (i.e. an exchange when no element is dropped), swap a pair of elements (i.e. 
an element from outside the cun^ent set is exchanged with an element from the set). With k>2 matroid 
constraints, we permit more general exchange operations, involving adding one element and dropping 
up to k elements. 

Note that the size of any local neighborhood is at most n^^^, which implies that each local step can 
be performed in polynomial time for a constant k. Let S denote a locally optimal solution. Next we prove 
a key lemma for this local search algorithm, which is used in analyzing our algorithm. Before presenting 
the lemma, we state a useful exchange property of mati'oids (see P31 ). Intuitively, this property states 
that for any two independent sets / and J, we can add any element of J to the set /, and kick out at most 
one element from / while keeping the set independent. Moreover, each element of / is allowed to be 
kicked out by at most one element of J. For completeness, a proof is given in Appendix lAl 

Theorem 1 Let A4 be a matroid and I,J(z ^{M) be two independent sets. Then there is a mapping 
TT : J \I ^ (I \ J) U {(j)} such that: 

1. {I\7r{b))U{b} £l{M)forallb£ J\I. 

2. |7r-i(e)| < lforalleeI\J. 
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Lemma 2 For a local optimal solution SandanyC G {k + l)-f{S) > f{SuC) + k- f{SnC). 

Additionally for k = 1, if S Ii is any locally optimal solution under only the swap operation, and 
C ell with \S\ = \C\, then 2 ■ f{S) > f{S U C) + f{S n C). 

Proof: The following proof is due to Jan Vondrak ll50l . Our original proof ||35]| was more complicated- 
we thank Jan for letting us present this simplified proof. 

For each matroid A4j {j € [k]), because both C,S £ Ij are independent sets, Theorem [T] implies a 
mapping ttj : C \ S {S \C) U {4>} such that: 

1. {S \ TTj{b)) U {b} G Ij for allbeC\S. 

2. |vr-^(e)| < Iforalle G S\C7. 

When k = 1 and \S\ = \C\, Corollary 39.12a from B31 implies the stronger condition that vri : C\S —>■ 
5 \ C is in fact a bijection. 

For each b £ C\S, let A,, = {tti (6) , • • • vTfc (6) }. Note that {S\Ab)U {b} G n'^=ilj for all 6 G C \ S. 
Hence {S \ Ab) U {6} is in the local neighborhood of S, and by local optimality under exchanges: 

f{S) > / ((5 \ Ab) U {b}) , V6 G C \ 5. (2) 

In the case k = I with |5| = \C\, these are only swap operations (because tti is a bijection here). 

By the property of mappings {tTj }^'^^, each element i G 5 \ C is contained in < A; of the sets 
{Ab I b G C\S}; and elements of SnC are contained in none of these sets. So the following inequalities 
are impUed by local optimality of S under deletions. 

(k-ui)- f{S)>{k-ni)- f{S\{i}), ViG5\C. (3) 

Note that these inequalities are not required when k = I and |5| = |Cj (then rii = k for alH G 5" \ C). 
For any 6 G C \ S", we have (below, the first inequality is submodularity and the second is from Q): 

f{S U {b}) - f{S) <f{{S\ At,) U W) -f{S\Ab)< f{S) -f{S\ At) 

Adding this inequality over all 6 G C \ S and using submodularity, 

f{SuC)-f{S)< [f{Su{b})-f{S)]< [/(5)-/(5\A)] 

bGC\S b£C\S 

Adding to this, the inequalities ©, i.e. < {k - rii) ■ [f{S) - f{S \ {i})] for alH G 5 \ C, 

f{s uc)- f{s) < Yl [/(^) - / \ ^^)] + E - • [/(^) - \ 

b£C\s ies\c 

X 

1=1 

where A = |C \ + J2ieS\c(^ ~ '^^) ^'^'^ {'^i}f=i some collection of subsets oi S \ C such that 
each i £ S\C appeai^s in exactly k of these subsets. Let s = jS"! and \S r\C\ = c; number the elements 
of 5 as {1, 2, • • • , s} = [s] such that S n C = {1, 2, • • • , c} = [c]. Then for any T C 5" \ C, by 
submodularity: f{S) - f{S \T) < J^per ifiM) - fiiP - M)]- Using this in ©, we obtain: 

A s 

f{suc)-f{s) < E E [/(M) - f(lP - 1])] = ^ E [/([^]) - /([^ - 1])] = ^•(/(^) - n c)) 

1=1 peTi i=c+l 
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Approximate Local Search Procedure B: 

Input: Ground set X of elements and value oracle access to submodular function /. 

1. Let {v} be a singleton set with the maximum value /({«}) and let S = {v}. 

2. While there exists the following delete or exchange local operation that increases the value of 
,f{S) by a factor of at least 1 + then apply the local operation and update S accordingly. 

• Delete operation on S. If e e S such that f{S \ {e}) > (1 + then S ^ S\ {e}. 

• Exchange operation on S. If d £ X \ S and G 5 U {(/)} (for 1 < i < fc) are such that 

(5 \ {e j) U {d} e X, for all ^ e [k] and \ {ei, • • • , Ck}) U {d}) > (1 + ^)f{S), 
fhsnS^iS\{ei,--- ,ek})U{d}. 



Figure 1: The approximate local search procedure. 



Algorithm A: 



1. Set = V. 

2. For i = 1, - ■ ■ ,k + l, do: 

(a) Apply the approximate local search procedure B on the ground set Vi to obtain a solution 
Si C Vi corresponding to the problem: 

max{/(5) : S £ Ci^^^Ij, S C V,}. (5) 

(b) Set V,+i = V \ S,. 

3. Return the solution corresponding to max{/(5i), • • • , f{Sk+i)}- 



Figure 2: Approximation algorithm for submodular maximization under k matroid constraints. 



The second last equality follows from S \ C = {c + I, ■ ■ ■ , s} and the fact that each element of S \ C 
appears in exactly k of the sets {Ti}^^^. The last equality is due to a telescoping summation. Thus we 
obtain (fc + 1) • f{S) > f{SuC) + k- f{S n C), giving the claim. 

Observe that when k = 1 and |5| = |C|, we only used the inequalities ^ from the local search, 
which are only swap operations. Hence in this case, the statement also holds for any solution S that 
is locally optimal under only swap operations. In the general case, we use both inequalities ^ (from 
exchange operations) and inequalities ^ (from deletion operations). ■ 

A simple consequence of Lemma |2] implies bounds analogous to known approximation factors |[39l 
l42l in the cases when the submodular function / has additional structure. 

Corollary 3 For a locally optimal solution S and any C G H^L^Zj the following inequalities hold: 

1- f{S)>f{C)/{k+l) if function f is monotone, 
2. fiS)>f{C)/k if function f is linear. 

The local search algorithm defined above could run for an exponential amount of time until it reaches 
a locally optimal solution. The standard approach is to consider an approximate local search. In Ap- 
pendix |Al we show an inequality (Lemma [T6l) analogous to Lemma[2]for approximate local optimum. 
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Theorem 4 Algorithm A in Figure |2] is a I . I n I 1 -I ) 'Approximation algorithm for maximizing a 
non-negative submodular function subject to any k matroid constraints, running in time n' 



0{k) 



Proof: Bounding the running time of Algorithm A is easy and we leave it to Appendix [A] Here, we 
prove the performance guarantee of Algorithm A. Let C denote the optimal solution to the original 
problem max{/(5) : 5 e n'^=ilj, S C V}. Let d = CnVi for each i £ [k + 1]; so Ci = C. 
Observe that Ci is a feasible solution to the problem dSjl solved in the ith iteration. Applying Lemma [T6] 
to problem ^ using the local optimum Si and solution Ci, we obtain: 

{l + e){k + l)- f{Si)> f{SiUC,) + k- fiSiHC,) yi<i<k + l, (6) 

Using f{S) > max^^^ f{Si), we add these k + 1 inequalities and simplify inductively as follows. 

Claim 5 For any 1 < I < k + 1, we have: 

k+l 

(1 + e){k + If ■ f{S) > (/-!)• f{C) + /(U^=i5p uCi)+ f(Si U Ci) 

i=l+l 

l-l k+l 

+ J2ik -i+p)-f{SpnCp) + k-Y, f{S^ n a). 

p=l i=l 

Proof: We argue by induction on /. The base case / = 1 is trivial, by just considering the sum of the 
k + l inequalities in statement Q above. Assuming the statement for some value 1 < / < A; + 1, we 
prove the corresponding statement for / + 1. 

{l + e){k + lf-f{S) > (/-!)• /(C) + /(U^=i5pUCi) 

fc+l l~l k+l 

+ J2 f {Si lid) + Y,ik- 1 +p)f{SpnCp) + k■Y,fis^nc,) 

i=l+l p=l i=l 

= (/-!)• fiC) + f{ul^,Sp U Ci) + fiSi+i U Q+i) 

fc+l l-l k+l 

+ fiSi u Ci) + - i+p)f{Sp nCp) + k ■Yf(s^ n Q) 

i=l+2 p=l i=l 

> (/-!)• /(C) + /(u;,t\SpUCi) + /(Q+i) 

fc+l l-l k+l 

+ Y f(Si u Ci) + - 1 + p)f{Sp nCp) + k-Y, fis^ n c,) 

i=l+2 p=l i=l 

I 

= (l-l)- f{C) + f{u%\Sp u Ci) + /(c+i) + Y fiSp n Cp) 

p=i 

k+l I k+l 

+ Y fi^i u Ci) + - ^ - 1 + p)fiSp nCp) + k- Y fis^ n d) 

i=l+2 p=l 1=1+1 

> i-f{C) + f{u'pt\SpUd) 

k+l I k+l 

+ J2 f(Si u d) + ^(A; - 1 - i+p)f{Sp nCp) + k- J2 fis^ n Q). 

i=l+2 p=l i=l+l 

The first inequality is the induction hypothesis, and the next two inequalities follow from submodu- 
larity using (ujj^iS'p n Cp) U C+i = C. ■ 
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Using the statement of Claim|5]when ^ = A: + 1, we obtain (1 + e)(/c + 1)^ • f{S) > k ■ f{C). ■ 

Finally, we give an improved approximation algorithm for symmetric submodular functions /, that 
satisfy f{S) = f{S) for all S (Z V . Symmetric submodular functions have been considered widely 
in the literature |[T7ll4TI . and it appears that symmetry allows for better approximation results and thus 
deserves separate attention. 

Theorem 6 There is a {jj^r^jj^j^i^ -approximation algorithm for maximizing a non-negative symmetric 
submodular functions subject to k matroid constraints. 

Proof: The algorithm for symmetric submodular functions is simpler. In this case, we only need to 
perform one iteration of the approximate local search procedure B (as opposed to /c + 1 in Theorem H]). 
Let C denote the optimal solution, and Si the result of the local search (on V). Then Lemma |2] implies: 

(1 + e){k + i)- f{Si) > f{Si uc) + k- f{Si n C) > f{Si uc) + f{Si n c). 

Because / is symmetric, we also have f{Si) = f{Si). Adding these two, 

(1 + e)(A: + 2) • f{Si) > f(S~i) + U C) + n C) > f{C \ Si) + /{Si n C) > f{C) . 
Thus we have the desired approximation guarantee. ■ 



3 Knapsack constraints 

Let f : 2^ be a submodular function, and w^, - ■ ■ , li;^ be A; weight- vectors corresponding to 

knapsacks having capacities Ci, • • • , respectively. The problem we consider in this section is: 

max{/(5) : ^ < Q, VI < i < A;, 5 C V}. (7) 

By scaling each knapsack, we assume that Cj = 1 for all z € [k]. We denote fmax = max{/(f) : v G 
V}. We assume without loss of generality that for every i ^ V, the singleton solution {i] is feasible 
for all the knapsacks (otherwise such elements can be dropped from consideration). To solve the above 
problem, we first define a fractional relaxation of the submodular function, and give an approximation 
algorithm for this fractional relaxation. Then, we show how to design an approximation algorithm for 
the original integral problem using the solution for the fractional relaxation. Let F : [0, 1]" M+, the 
fractional relaxation of f, be the 'extension-by-expectation' of /, i.e., 

F{x) = fiS) ■ ^iGSXi ■ nj0s(l - xj). 

scv 

Note that F is a multi-linear polynomial in variables xi, • • • , Xn, and has continuous derivatives of all 
orders. Furthermore, as shown in Vondrak P9ll , for all i,j € V, q^q^ F < everywhere on [0, 1]" ; we 
refer to this condition as continuous submodularity. 
Due to lack of space, many proofs of this section appear in Appendix iBl 

Extending function / on scaled ground sets. Let Sj € Z+ be arbitrary values for each i ^ V. 
Define a new ground-set U that contains Sj 'copies' of each element i € so the total number of 
elements in U is X^jgy Sj- We will denote any subset T ofU as T = Ujgyli where each Tj consists of all 

copies of element i from T. Now define function g : 2^ ^ as (/(UigyTi) = F{- ■ ■ , -1^, • • • ). 

Our goal is to prove the useful lemma that g is submodular. In preparation for that, we first establish 
a couple of claims. The first claim is standard, but we give a proof for the sake of completeness. 
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Claim 7 Suppose Z : D — > R has continuous partial derivatives everywhere on convex V C with 
'§^{y) ^ O/or all y ^ V and i (z V. Then for any oi, 02 G V with ai < 02 coordinate-wise, we have 
l{ai) > l{a2). 

Next, we establish the following property of the fractional relaxation F. 

Claim 8 For any a,q,d £ [0,1]" with q + d £ [0,1]" and a < q coordinate-wise, we have F{a -\- d) — 
F{a) > F{q + d)-F{q). 

Using the above claims, we are now ready to state and prove the lemma. 

Lemma 9 Set function g is a submodular function on ground set U. 

Solving the fractional relaxation. We now argue how to obtain a near-optimal fractional fea- 
sible solution for maximizing a non-negative submodular function over k knapsack constraints. Let 
w^, - ■ ■ ^w^ denote the weight- vectors in each of the k knapsacks such that all knapsacks have capacity 
1. The problem we consider here has additional upper bounds {uj € [0, l]}"=i on variables: 

max{F(?/) : ■y<l Vs € [A;], < y.j < Ui Vi G V}. (8) 

Denote the region U := {y : ^ < yi < Ui'ii £ V}. We define a local search procedure to solve 
problem ([8]l. We only consider values for each variable from a discrete set of values in [0, 1], namely 
Q = {p ' C ^pSN, 0<p<^} where Q = Let e > be a parameter to be fixed later. At any 
step with current solution y G [0, 1]", the following local moves are considered: 

• Let A,D Q [n] with \A\, \D\ < k. Decrease the variables y{D) to any values in Q and increase 
variables y{A) to any values in Q such that the resulting solution y' still satisfies all knapsacks and 
y' G U. If F{y') > (1 + e) • F{y) then set y ^ y' . 

Note that the size of each local neighborhood is rf^^\ Let a be the index corresponding to maxjuj • 
/({i}) : i G V}. We start the local search procedure with the solution y^ having ?/o(fl) = Ua and zero 
otherwise. Observe that for any x G W^, F{x) < Yl^=i ' /({O) ^n-Ua- /({a}) = n ■ F{yo). Hence 
the number of iterations of local search is 0(i log n), and the entire procedure terminates in polynomial 
time. Let y denote a local optimal solution. We first prove the following based on the discretization Q. 

Claim 10 Suppose a,(3 £ [0, 1]" are such that each has at most k positive coordinates, y' := y—a-\-j3 G 
U, and y' satisfies all knapsacks. Then F{y') < (1 + e) • F{y) + -^fmax- 

For any x,y € M", we define xVy and xAy by {x\/y)j := m.ax{xj,yj) and {xAy)j := min(xj, yj) 
for j G [n] . 

Lemma 11 For local optimal y & U DQ^ and any x £ LI satisfying the knapsack constraints, we have 
(2 + 2n • e) • F{y) > F{y A x) + F{y V x) - ^ ■ fmax- 

Proof: For the sake of this proof, we assume that each knapsack s G [k] has a dummy element (which 
has no effect on function /) of weight 1 in knapsack s (and zero in all other knapsacks), and upper-bound 
of 1. So any fractional solution can be augmented to another of the same F- value, while satisfying all 
knapsacks at equality. We augment y and x using dummy elements so that both satisfy all knapsacks at 
equality: this does not change any of the values F{y), F{y A x) and F{y V x). Let y' = y — {y Ax) and 
x' = X — {y A x). Note that for all s G [k], • y' = ■ x' and let Cg = • x' . We will decompose y' 
and x' into an equal number of terms as y' = at and x' = [5t such that the as and /3s have small 
support, and w'^ ■ at = w'^ ■ (3t for all t and s G [k] . 

1. Initialize t ^ 1, 7 ^ 1, x" ^ x', y" ^ y' . 

2. While 7 > 0, do: 
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(a) Consider LP^ := > : z ■ w" = Cs, Vs G [k]} where the variables are restricted to indices 
i G [n] with x'/ > 0. Similarly LPy := {z > : z • ly* = c,, Vs e [k]} where the variables are 
restricted to indices i E [n] with i// > 0. Let u G LPx and v G LPy be extreme points. 

(b) Set Si ~ max{x : X • m < x"} S2 = max{x : X ' " ^ 2/"}' ™d ^ = niin{(5i, (52}- 

(c) Set /3t <— (5 • u, at ^ (5 • 7 ^ 7 — (5, x" ^ a;" — /?t, and y" ^ y" — at. 

(d) Set t ^ i + 1. 

We first show that this procedure is well-defined. In every iteration, 7 > 0, and by induction 
■ x" = ■ y" = J ■ Cs for all s £ [k]. Thus in stepl2al LP^ (resp. LPy) is non-empty: x" (resp. 
y" /j) is a feasible solution. From the definition of LP^ and LPy it also follows that (5 > in step l2b] and 
at least one coordinate of x" or y" is zeroed out in step|2c] This implies that the decomposition procedure 
terminates in r < 2n steps. At the end of the procedure, we have decompositions x' = X]t=i A 
y' = Furthermore, each at (resp. (it) corresponds to an extreme point of LPy (resp. LP^) 

in some iteration: hence the number of positive components in any of {at^ l3tYt=i most k. Finally 
note that for all t € [r], ■ at = • [3t for all knapsacks s G [k]. 

Observe that Claim [TOl applies to y, at and I3t (any t € [r]) because each of aj, fit has support-size 
k, and y — at + Pt £ ^ and satisfies all knapsacks with equality. Strictly speaking. Claim [TOlrequires the 
original local optimal y, which is not augmented with dummy elements. However even y — at + (it 
and satisfies all knapsacks (possibly not at equality), and the claim does apply. This gives: 

F{y -at + (it) < (1 + 6) • F{y) + ^ Vt e [r] . (9) 

Let M G Z+ be large enough so that Mat and M(it are integral for all t G [r]. In the rest of the 
proof, we consider a scaled ground-set U containing Af copies of each element in V. We define function 
g : 2^ ^ M_i_ as (7(UjgyTj) = F{- ■ ■ , • ■ ■ ) where each Tj consists of copies of element i G V. 
Lemma |9] implies that g is submodular. CoiTcsponding to y we have a set P = Ui^yPi consisting of the 
first \Pi\ = M -yi copies of each element i G V. Similarly, x corresponds to set Q = Ui^vQi consisting 
of the first \Qi\ = M ■ x-i copies of each element i G V. Hence P H Q (resp. P U Q) con^esponds to 
X Ay (resp. x\J y) scaled by M. Again, P\Q (resp. Q\P) corresponds to y' (resp. x') scaled by M. 
The decomposition of y' from above suggests disjoint sets {^jjj'^]^ such that VJt-^t = P\Q', i-C- each 
At corresponds to at scaled by M. Similai^ly there are disjoint sets {Bt}l^i such that UtBt = Q \ P. 
Observe also that g{{P \ At) U Bt) = F{y — at + (it), so ^ corresponds to: 

g{{P \ At) U Bt) < (1 + e) • g{P) + ^ Vt G [r] . (10) 

Adding all these r inequalities to g{P) = g{P), we obtain (r + e • r + l)g{P) + -^fmax > 9{P) + 
J2t=i 9i{P\^t)^Bt). Using submodularity of g and the disjointness of families {At}f-i and {Bt}f-i, 
we obtain (r + e • r + 1) • g{P) + ^f^ax > (r - 1) • g{P) + g{P U Q) + g{P n Q). Hence 
(2 + e • r) • g{P) > g{P VJ Q) + g{P CiQ) — -^fmax- This implies the lemma because r < 2n. ■ 

Theorem 12 For any constant 6 > 0, there exists a — 6) -approximation algorithm for problem ^ 
with all upper bounds Ui = \ (for all i £ V). 

Rounding the fractional knapsack. In order to solve the (non-fractional) submodular maximization 
subject to k knapsack constraints, we partition the elements into two subsets. For a constant parameter 
6, we say that element e € F is heavy if w'^{e) > 6 for some knapsack s G [k]. All other elements 
are called light. Since the number of heavy elements in any feasible solution is bounded by ^, we can 
enumerate over all possible sets of heavy elements and obtain the optimal profit for these elements. For 
light elements, we show a simple randomized rounding procedure (applied to the fractional solution 
from ([SI)) that gives a — e) -approximation for the light elements. Combining the enumeration method 
with the randomized rounding method, we get a — e) -approximation for submodular maximization 
subject to k knapsack constraints. The details and proofs of this result are left to Appendix ICl 
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4 Improved Bounds under Partition Matroids 



The improved algorithm for partition matroids is again based on local search. In the exchange local 
move of the general case (Section |2]|, the algorithm only attempts to include one new element at a time 
(while dropping up to k elements). Here we generalize that step to allow including p new elements 
while dropping up to (/c — 1) • p elements, for some fixed constant p > I. We show that this yields an 
improvement under partition matroid constraints. Given a current solution S € Hj^^Ij, the local moves 
we consider are: 

• Delete operation. If e £ S such that f{S \ {e}) > f{S), then S {e}. 

• Exchange operation. For some q < p, if di, - ■ ■ ,dq G V \ S and G 5 U {(/)} (for 1 < i < 
(A; - 1) • q) are such that: (i) S' = {S \ {ei : 1 < i < {k - l)q}) U {di, • • • , G Ij for all 
j G [k], and (ii) f{S') > f {S), then S ^ S'. 

The main idea here is a strengthening of Lemma[2l Missing proofs of this section are in Appendix iDl 

Lemma 13 For a local optimal solution S and any C G Hj^ilj, we have k ■ f{S) > (1 — p • f{S U 

c) + {k-i)-f{snc). 

Proof: We use an exchange property (see Schrijver P31I '). which implies for any partition matroid A4 
and C, S* G I{M) the existence of a map tt : C \ S ^ {S \C) U {(j)} such that 

1. {S \ {7T{b) : 6 G T}) U T G I{M) for all T C C7 \ S. 

2. |7r-i(e)| < Iforalle G S\C7. 

Let vTj denote the mapping under partition matroid M.j (for 1 < j < k). 

Combining partition matroids Mi and A^2- We use vri and tt2 to construct a multigraph G on vertex 
set C \ S and edge-set labeled by = 7ri(C \ S") n 7r2(C \ 5) C S\C with an edge labeled a e E 
between e, / G C \ 5 iff 7ri(e) = vr2(/) = a or iT2{e) = 7ri(/) = a. Each edge in G has a unique 
label because there is exactly one edge (e, /) con^esponding to any a G E. Note that the maximum 
degree in G is at most 2. Hence G is a union of disjoint cycles and paths. We index elements of C \ S 
in such a way that elements along any path or cycle in G are consecutive. For any g G {0, • • • ,p — 1}, 
let Rq denote the elements of C \ S" having an index that is not q modulo p. It is clear that the induced 
graph G[Rq] for any G [p] consists of disjoint paths/cycles, each of length at most p. Furthermore each 
element of C \ 5 appears in exactly p — 1 sets among {Rq}^~Q. 

Claim 14 For anyqe{0,--- ,p- 1}, k ■ f{S) > f{S U Rq) + {k - I) ■ f{S n G). 

Adding the p inequalities given by Claim[ll we getpA;-/(S) > ^^"q f{SURq)+p{k-l)-f{SriG). 

Note that each element of G\S is missing in exactly 1 set {SURq}^~l, and elements of SCiG are missing 

in none of them. Hence an identical simplification as in Lemma|2]gives X^q=o[/(»S'UC) — f{SURq)] < 
f{SuC)-f{S). Thus, 

{pk - 1) • f{S) > (p - 1) • f{s uG)+ p{k - 1) • f{s n G), 

which impUes k ■ f{S) > (1 - i) • f{S UG) + {k-l)- f{S n C), giving the lemma. ■ 

Theorem 15 For any k > 2 and fixed constant e > 0, there exists a ^ ^^ -approximation al- 

gorithm for maximizing a non-negative submodular function over k partition matroids. This bound 
improves to -j^for monotone submodular functions. 

Acknowledgment: The proof of Lemma |2] presented in this paper is due to Jan Vondrak. Our original 
proof [351 was more complicated — we thank Jan for letting us present this simplified proof. 
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A Missing proofs from Section |2] 

Proof of Theorem [ij We proceed by induction on t = \J\I\. If t = 0, there is nothing to prove. 
Suppose there is an element b £ J \ I with / U {b} € I{A4). In this case we apply induction on 
/ and J' = J \ {b} (where \J' \ I\ = t — 1 < t). Because I \ J' = / \ J, we obtain a map 
tt' : J' \ I {I \ J) U {(/>} satisfying the two conditions. The desired map vr is then 7r(6) = (p and 
7r(6') = TT'{b') for all 6' G J \ / \ {b} = J'\I. 

Now we may assume that / is a maximal independent set in / U J. Let A4' C M denote the matroid 
A4 truncated to / U J; so / is a base in A4'. We augment J to some base J ^ J in A4' (because any 
maximal independent set in A4' is a base). Then we have two bases / and J in A4'. Theorem 39.12 
from [i45] implies the existence of elements 6 G J \ / and e £ I \ J such that both ( J \ 6) U {e} and 
(/ \ e) U {b} are bases in M'. Note that J' = {J\{b})U {e} C {J\ {b}) U{e} £ M.By induction on 
/ and J' (because | J' \ / | = t — 1 < t) we obtain map tt' : J'\I I\J' satisfying the two conditions. 
The map vr is then 7r(6) = e and vr(6') = vr'(6') for all 6' G J \ / \ {6} = J' \I. The first condition on 
vr is satisfied by induction (for elements J \ I \ {b}) and because (/ \ e) U {b} G M. (see above). The 
second condition on vr is satisfied by induction and the fact that e ^ I \ J'. ■ 

Lemma 16 For an approximately locally optimal solution S (in procedure B) and any C G R^^j^Xj, 
(1 + €){k + 1) • f{S) > f{S U C) + k ■ f{S n C) where e > a parameter defined in the algorithm 
description. Additionally for k = 1, if S £ Ii is any locally optimal solution under only the swap 
operation, and C £ Ii with \S\ = \C\, then 2(1 + e) • > f{S U C) + f{S n C). 

Proof: The proof of this lemma is almost identical to the proof of the Lemma [2] the only difference 
is that left-hand sides of inequalities Q and inequalities ^ are multiplied by 1 + Therefore, after 
following the steps in Lemma|2l we obtain the inequality: 

(A; + 1 + 4a) • f{S) > f{S UC) + k- fis n C). 

Since \ < {k + l)n (see Lemma|2l) and we may assume that >> {k + l)n, we obtain the lemma. ■ 

Running time of Algoritiim A (Theorem|4ll Here we describe a missing part of the proof of Theorem|4] 
about the running of Algorithm A. The parameter e > in Procedure B is any value such that - is 
at most a polynomial in n. Note that using approximate local operations in the local search procedure 
B (in Figure [T]) makes the running time of the algorithm polynomial. The reason is as follows: one 
can easily show that for any ground set X of elements, the value of the initial set S = {f } is at least 
Opt(X)/n, where Opt(X) is the optimal value of problem ([T|l restricted to X. Each local operation 
in procedure B increases the value of the function by a factor 1 + Therefore, the number of local 

operations for procedure B is at most logi_|_ ^ ^pt(''^)'' = 0{-n^ log n), and thus the running time of the 

n 

whole procedure is - ■ iiP^^\ Moreover, the number of procedure calls of Algorithm A for procedure B 
is polynomial, and thus the running time of Algorithm A is also polynomial. 
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B Missing Proofs from Section |3] 



Proof of Claim|7} Consider the line from oi to 02 parameterized by t G [0,1] asy(t) := ai+t{a2 — ai)- 
Observe that all points on this line are in V (because P is a convex set). At any t € [0, 1], we have: 



dljyit)) 
dt 



aHy{t)) ayj{t) 



dt 



E 



(a2(j) -«i(j)) <0. 



Above, the first equality follows from the chain rule because I is differentiable, and the last inequality 
uses the fact that 02 — ai > coordinate-wise. This completes the proof of the claim. ■ 

Proof of Claim lit Let P = {y G [0, 1]" : y + d e [0, 1]"}. Define function h : V ^ 



M+ as 

h{x) := F{x + d) — F{x), which is a multi-linear polynomial. We will show that ■§^{a) < for all 
i ^ V, at every point a ^ V. This combined with Claim|7]below would imply h{a) > h{q) because 
a < q coordinate-wise, which gives the claim. 

In the following, fix an i G F and denote i^'(y) = §§-{y) for any y G [0, 1]". To show ^(a) < 
for a G P, it suffices to have -F/(a + d) — Fl{a) < 0. From the continuous submodularity of F, for 



every j G V we have jf-^{y) 



(y) < for all y G [0, 1]". Then applying Claim |7] to Fl (a 
multi-linear polynomial) implies that Fl{a + d) — Fl{a) < 0. This completes the proof of Claim[8] ■ 

Proof of Lemma IS To show submodularity of g, consider any two subsets P = UjgyPj and Q = 
UjgyQi of U, where each Pi (resp., Qi) are copies of element i £ V. We have P HQ = Ui£v{Pi H Qi) 
and P U Q = Ui^viPi U Qi). Define vectors p, q,a,b G [0, 1]" as follows: 



Pi 



\3\ 

Si 



\Qi 



\Pi n Qi 



\Pi u Qi 



yi G V. 



Si 



It is clear that p + q = a + b and d := p — a > 0. Submodularity condition on g at P,Q requires 
g{P)+g{Q) > g{PnQ)+g{PUQ). But by the definition of 5-, this is equivalent to F{a + d)-F{a) > 
F{q + d) — F{q), which is true by Claim[8] Thus we have established the lemma. ■ 

Proof of Claim [lOl Let z < y' be the point mU r\Q^ that minimizes Y^^=i iy'i ~ ^i)- Note that z is a 
feasible local move from y: it lies in Q^, satisfies all knapsacks and the upper-bounds, and is obtainable 
from y by reducing A; variables and increasing /c others. Hence by local optimality ^(z) < (l + e)-F(y). 
By the choice of z, it follows that |zj — y-| < C, for all i £ V. Suppose B is an upper bound on all 



first partial derivatives of function F on U: i.e. 
F has continuous derivatives, we obtain 



dF{x) 
dxi 



< B for alH G F and x £U. Then because 



\F{z)-F{y')\<Y,B 



y[\ 



< nBC < 2n2/, 



2f .r<tl 

max S — 



4n2 



^ , and the second to last inequality 



Above /max = max{/(t') : v G F}. The last inequality uses ^ — g^. 
uses B < 2n ■ fmax which we show next. Consider any x G [0, 1]"^ and i £ V. We have 



dF{x) 



dxi 



< 



^ [fiS U {i}) - f{S)] ■ UaeSXa • U,^S^\i{l - Xt) 
SC[n]\{i} 

max [f{S U {i}) + f{S)] < 2n ■ fmax ■ 



Thus we have F{y') < (1 + e) • F{y) + ^fmax- ■ 

Proof of Theorem [I2t Because each singleton solution {i} is feasible for the knapsacks and upper 
bounds are 1, we have a feasible solution of value fmax- Choose e = The algorithm runs the 
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fractional local search algorithm (with all upper bounds 1) to get locally optimal solution yi G [0, 1]". 
Then we run another fractional local search, this time with each variable i £ V having upper bound 
Ui = 1 — yi{i); let y2 denote the local optimum obtained here. The algorithm outputs the better of the 
solutions 7/1,2/2, and value fmax- 

Let X denote the globally optimal fractional solution to (HJl, where upper bounds are 1. We will 
show {2 + 5) ■ {F{yi) + F{y2)) > F{x) — fmax/n, which would prove the theorem. Observe that 
x' = X — {x A yi) is a. feasible solution to the second local seaixh. Lemma [TT] implies the following for 
the two local optima: 

{2 + 6)-F{yi) > F{xAy,)+F{xVyi) -^"^"^ 



(2 + 5)-F(y2) > F{x' Ay2)+F{x'Vy2) 



2n ' 

fmax 



2n 

We show that F{x f\yi) + F{x V yi) + F{x' V 7/2) > F{x), which suffices to prove the theorem. For 
this inequality, we again consider a scaled ground-set U having M copies of each element in V (where 
M € is large enough so that Mx, Myi, My2 are all integral). Define function g : 2^ —>■ ]R+ as 
g{Ui^vTi) = F{- ■ ■ , • • • ) where each Ti consists of copies of element i £ V. Lemma |9] implies 
that g is submodular. Also define the following subsets of U: A (representing yi) consists of the first 
Myi{i) copies of each element i £ V, C (representing x) consists of the first Mx{i) copies of each 
element i £ V, and B (representing 7/2) consists of My2{i) copies of each element i £ V (namely the 
copies numbered Myi{i) + 1 through Myi{i) + My2{i)) so that Ar\ B = cj). Note that we can indeed 
pick such sets because yi + 1/2 < 1 coordinate-wise. Also we have the following correspondences via 
scaling: 

AnC = xAyi, AUC = x\J yi, {C \ A) U B = x' \/ y2 . 

Thus it suffices to show g{A D C) + g{A U C) + g{{C \ A) U B) > g{C). But this follows from 
submodularity and non-negativity of g: 

g{A n C) + g{A U C) + g{{C \A)UB)> g{A nC)+ g{C \ A) + g{C U Au B) > g{C). 

Hence we have the desired approximation for the fractional problem ([S]). ■ 



C Rounding the fractional solution under knapsack constraints 

Fix a constant 77 > and let c = We give a — r/) -approximation for submodular maximization over 
k knapsack constraints, which is problem ([7]). Define parameter 6 = j;^^- We call an element e £ V 
heavy if w'^{e) > 6 for some knapsack i G [k]. All other elements are called light. Let H and L denote 
the heavy and light elements in an optimal integral solution. Note that \H\ < k/5. Hence enumerating 
over all possible sets of heavy elements, we can obtain profit at least f[H) in rP^^/^) time, which is 
polynomial for fixed k. We now focus only on light elements and show how to obtain profit at least 
I • f{L). Later we show how these can be combined into an approximation algorithm for problem dV]). 
Let Opt > f{L) denote the optimal value of the knapsack constrained problem, restricted to only light 
elements. 

Algorithm for light elements. Restricted to light elements, the algorithm first solves the fractional 
relaxation ([8]) with all upper bounds 1, to obtain solution x with F{x) > (| — |) • Opt, as described in 
the previous subsection (see Theorem [12]). Again by adding dummy light elements for each knapsack, 
we assume that fractional solution x satisfies all knapsacks with equality. Fix a parameter e = and 
pick each element e into solution S independently with probability (1 — e) • Xg. We declare failure if S 
violates any knapsack and claim zero profit in this case (output the empty set as solution). Clearly this 
algorithm always outputs a feasible solution. In the following we lower bound the expected profit. Let 
a{S) := max{7i;*(5) : i G [k]}. 
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Claim 17 For any a > 1, Pr{a[S) >a\<k- e"™*^". 

Proof: Fixing a knapsack i ^ [k], we will bound Pr[w'^{S) > a]. Let denote the binary random 

variable which is set to 1 iff e G S, and let Yg = j^^ Xg. Because we only deal with light elements, each 
Yf, is a [0, 1] random variable. Let Zj := Y^, then = By scaling, it suffices to upper bound 

Pr [Zi > a(l + e)E[Zi]]. Because the Ye are independent [0, 1] random variables, Chernoff bounds |[37l 
imply: 

Pr [Zi > a(l + e)E[Z,]] < e~E{z,\-^t^ 12 < g-a.V45 ^ ^-cak\ 
Finally by a union bound, we obtain Pr{a{S) >a\< Pr{w\S) >a\<k- e"™*^". ■ 

Claim 18 For any a > 1, max{/(5) : a{S) < a + 1} < 2(1 + b)k{a + 1) • Opt 

Proof: We will sho w that for any set 5 with a (S) < a+l,/(S') < 2(l+5)A;(a+l)-Opt, which implies 
the claim. Consider partitioning set S into a number of smaller parts each of which satisfies all knapsacks 
as follows. As long as there are remaining elements in S, form a group by greedily adding 5-elements 
until no more addition is possible, then continue to form the next group. Except for the last group formed, 
every other group must have filled up some knapsack to extent \ — b (otherwise another light element 
can be added). Thus the number of groups partitioning S is at most ^'^'^^^^ + 1 < 2fc(a + !)(! + S). 
Because each of these groups is a feasible solution, the claim follows by the subadditivity of /. ■ 

Lemma 19 The algorithm for light elements obtains expected value at least (^ — ?/) • Opt. 

Proof: Define the following disjoint events: Aq := {a{S) < 1}, and Ai := {I < a{S) < 1 + /} for 
any / G N. Note that the expected value of the algorithm is ALG = E[f{S) \ Aq] ■ -Pr[Ao]. We can 
write: 

Fix) = E[fiS)] = E[fiS) I Ao]-Pr[Ao]+^E[fiS) | Ai]-Pr[Ai] = AlG+^E[fiS) \ Ai]-Pr[Ai]. 

i>i i>i 

For any / > 1, from ClaimOwe have Pr[Ai] < Pr[a{S) > I] < k ■ e'"^^ . From Claim[l8] we have 
E[f{S) I Ai] < 2(1 + 5)k{l + 1) • Opt. So, 

E[f{S) I Ai] ■ Pr[Ai] < k ■ e"""^" • 2(1 + 6)k{l + 1) • Opt < 8 • Opt • Ik"^ ■ e-^^''\ 

Consider the expression Yl,i>i ' e""^"^^ < Ylt>i ^ ' — c' large enough constant c. Thus: 

ALG = F{x) - E[f{S) I Ai] ■ Pr[Ai] > F{x) - 8 • Opt ^ /A: • e~^^'' > F{x) - -Opt. 

i>i i>i ^ 

Because = ^ and F{x) > (| — ^) • Opt from Theorem [12] we obtain the lemma. ■ 

Tlieorem 20 For any constant r] > 0, there is a (^ — if)-approximation algorithm for maximizing a 
non-negative submodular function over k knapsack constraints. 

Proof: As mentioned in the beginning of this subsection, let H and L denote the heavy and light 
elements in an optimal integral solution. The enumeration algorithm for heavy elements produces a 
solution of value at least f{H). Lemma [T9] implies that the rounding algorithm for light elements 
produces a solution of expected value at least (^ — ??) • f{L). By subadditivity, the optimal value 
f{H U L) < f{H) + f{L). The better of the two solutions (over heavy and light elements respectively) 
found by our algorithm has value: 

max{/(if), (i - r?) . /(L)} > 1 • /(H) + 1 • (1 - r?) • /(L) > (^ - r?) • /{H U L). 
This implies the desired approximation guarantee. ■ 



16 



D Missing Proofs from Section |4] 



Proof of Claim [TH The following arguments hold for any g € [p] , and for notational simplicity we 
denote R = Rq Q C\S. Let {-Dzjf^^ denote the vertices in connected components of G[R], which form 
a partition of R. As mentioned above, \Di\ < p for all / G [t]. For any I G [t], let Ei denote the labels 
of edges in G incident to vertices Di. Because {DiYi^^ distinct connected components in G[R], 
{Ei}j^-^ are disjoint subsets of E <Z S\C. Consider any I e [t]: we claim Si = {S\Ei)\J Di e Ii n J2. 
Note that D {7ri(6) : b € A}and£;, D {iT2{b) : b G A}- Hence S", C {S\{TTi{b) : b G Di})UDi 
for i = 1,2. But from the property of mapping tTj (where i = 1, 2), {S \ {71^(1?;)) U € X,;. This 
proves that Si eliH I2 for all I e [t]. 

From the properties of the maps ttj for each partition matroid Mj, we have {S \ TTj{Di)) U € Ij 
for each 3 < j < k. Thus the following sets are independent in all matroids Mi, - ■ ■ ,M.k'- 

{S \ (u^^^TTjiDi) u El)) uDi yie[t]. 

Additionally, because \Di\ < p and \{Uj^^TTj{Di) U Ei\ < {k — 1) ■ p, each of the above sets are in 
the local neighborhood of 5. But local optimality of S implies: 

fiS) > f{{S\iu';^^7r,{Di)UEi))uDi) VZ G [t]. (11) 

Recall that {Ei} are disjoint subsets of S* \ C. Also each element i G 5" \ C is missing in the right-hand 
side of nj < A; — 1 terms (the vr^s are 'matchings' onto S \C). Using local optimaUty under deletions, 
we have the inequaUties: 

{k-1- n,) ■ f{S) >{k-l- 71,) • f{S \ {i}) Vi G 5 \ C. (12) 

Now, proceeding as in the simplification done in Lemma|2] (using disjointness of {-D;};=i)> we obtain: 

f{su (uLiA)) - f{S) <{k-i)- {f{S) - f{s n c)) 

Noting that uj^iDi = R, we have the claim. ■ 

Proof of Theorem [lit We set p = 1 + [^] . The algorithm for the monotone case is just the local 
seaixh procedure with p-exchanges. Lemma [13] applied to local optimal S and the global optimal C 
implies f{S) > - 4) • f{S U C) > - ^) • /(C) (by non-negativity and monotonicity). From 
the setting of p, S is ak + e approximate solution. 

For the non-monotone case, the algorithm repeats the p-exchange local seaixh k times as in Theo- 

remlH If C denotes a global optimum, an identical analysis yields ^1 + k'^ 'fiS) > — 1) • f{C). 
This uses the inequalities 



P 



p — 1 



k ■ f{Si) > fiSi U Gi) + ik-l)- fiSi DC) VI < i < k. 



where Si denotes the local optimal solution in iteration i G {!,■■■ ,k} and Gi = G \ U*^\S'j. Using 
the value of 5" is a ( A: + 1 + + e ) -approximate solution. Observe that the algorithm has running 



time n'^C^A). ■ 

We note that the result for monotone submodular functions is the first improvement over the greedy 
-approximation algorithm |[39l . even for the special case of partition matroids. It is easy to see that 
the greedy algorithm is a ^-approximation for modular functions. But it is only a ^^-j- -approximation 
for monotone submodulai^ functions. The following example shows that this bound is tight for every 
k > 1. The submodular function / is the coverage function defined on a family T of sets. Consider a 
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ground set E = {e : 0<e<p(A; + l) + l}of natural numbers (for p >2 ai^bitrarily large); we define 
a family T = {Si : < i < k} U {Ti, T2} of A; + 3 subsets of E. We have 5o = {e : < e < p}, 
Ti = {e : < e < p - 1}, T2 = {p}, and for each I < i < k, Si = {e : p-i + l<e<p-{i + 1)}. 
For any subset S C f{S) equals the number of elements in E covered by 5; / is clearly monotone 
submodular. We now define k partition matroids over T: for 1 < j < k, the j*^ partition has {5o, Sj} 
in one group and all other sets in singleton groups. In other words, the partition constraints require that 
for every 1 < j < k, at most one of and Sj be chosen. Observe that {Si : 1 < i < A;} U {Ti, T2} is 
a feasible solution of value \E\ = p{k + 1) + 1. However the greedy algorithm picks So first (because 
it has maximum size), and gets only value p + I. 

E Matroid Base Constraints 

A base in a matroid is any maximal independent set. In this section, we consider the problem of maxi- 
mizing a non-negative submodular function over bases of some matroid M.. 

max{/(S) : S e B{M)} . (13) 

We first consider the case of symmetric submodular functions. 

Theorem 21 There is a — e) -approximation algorithm for maximizing a non-negative symmetric 
submodular function over bases of any matroid. 

Proof: We use the natural local search algorithm based only on swap operations. The algorithm starts 
with any maximal independent set and performs improving swaps until none is possible. From the 
second statement of Lemma |2j if 5 is a local optimum and C is the optimal base, we have 2 ■ f{S) > 
f{SuC) + f{S n C). Adding to this inequality, the fact /(5) = f{S) using symmetry, we obtain 
3 • f{S) > f{S UC) + f(S) + f{S n C) > f{C \S) + f{S nC)> f{C). using an approximate local 
search procedure to make the running time polynomial, we obtain the theorem. ■ 

However, the approximation guarantee of this algorithm can be arbitrarily bad if the function / is 
not symmetric. An example is the directed-cut function in a digraph with a vertex bipartition {U, V) 
with |?7| = \V\ = n, having t ^ 1 edges from each [/-vertex to V and 1 edge from each ^/-vertex to 
U. The matroid in this example is just the uniform matroid with rank n. It is clear that the optimal base 
is U; on the other hand F is a local optimum under swaps. 

We aie not aware of a constant approximation for the problem of maximizing a submodular function 
subject to an arbitrary matroid base constraint. For a special class of matroids we obtain the following. 

Theorem 22 There is a — -approximation algorithm for maximizing any non-negative submodular 
function over bases of matroid A4, when A4 contains at least two disjoint bases. 

Proof: Let C denote the optimal base. The algorithm here first runs the local search algorithm using 
only swaps to obtain a base Si that satisfies 2 • f{Si) > f{Si U C) + f{Si n C), from Lemma|2l Then 
the algorithm runs a local search on F \ S*! using both exchanges and deletions to obtain an independent 
set 52 C F \ 5i satisfying 2 • /{Si) > f{S2 U (C \ Si)) + f{S2 n (C \ Si)). Consider the matroid 
M' obtained by contracting S2 in M. Our assumption implies that A4' also has two disjoint bases, say 
Bi and B2 (which can also be computed in polynomial time). Note that S2 U Bi and ^2 U B2 are bases 
in the original matroid M. The algorithm outputs solution S which is the better of the three bases: Si, 
S2 U Bi and S2UB2. We have 

6/(5) > 2/(5i) + 2(/(52Ui?i) + /(52Ui?2)) > 2/(5i) + 2/(52) 
> /(5i U C) + /(5i n C) + /(52 U (C \ 5i)) > /(C). 

The second inequality uses the disjointness of Bi and B2. ■ 
A consequence of this result is the following. 



18 



Corollary 23 Given any non-negative submodular function / : 2^ ^ and an integer < c < \V\, 
there is a {■^ — e)-approximation algorithm for the problem m.a\{f{S) : S^V, 15*1=0}. 

Proof: If c < |y 1/2 then the assumption in Theorem (l22l ) holds for the rank c uniform matroid, and the 
theorem follows. We show that c < \V\/2 can be ensured without loss of generality. Define function 
g : 2^ M+ as g{T) = f{V \ T) for all T C y. Because / is non-negative and submodular, so is g. 
Furthermore, max{/(5) : S" C y, |5| = c} = max{g(T) : T C F, |r| = \V\ - c}. Clearly one of 
c and \ V\ — c is at most |l^|/2, and we can apply Theorem |22] to the corresponding problem. ■ 
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